Verification system, verification method, and information storage medium

ABSTRACT

A transformed image generation unit  32  generates a transformed image by transforming an oblique image. A target region specifying unit  36  specifies a target region indicating the target in the front image based on a feature extracted from a typical portion of a document written on a target in a sample image and a feature extracted from at least a part of the front image. A verification unit  38  determines whether a target shown in the oblique image is the same as a target shown in the front image by verifying the target region in the front image and a region in the transformed image associated with the target region.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure contains subject matter related to that disclosedin Japanese Priority Patent Application JP 2020-162139 filed in theJapan Patent Office on Sep. 28, 2020, the entire contents of which arehereby incorporated by reference.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a verification system, a verificationmethod, and an information storage medium.

Description of the Related Art

Scale invariant feature transform (SIFT) which is a type of distinctiveimage feature quantity is described in “Distinctive Image Features fromScale-Invariant Keypoints” by David G. Lowe, International Journal ofComputer Vision, 2004, Jan. 5, 2004.

There is known a technology of an electronic know your customer (eKYC)for performing know your customer of a user online based on an image ofa know your customer document such as a driver license transmitted fromthe user via the Internet. WO 2020/008628 discloses a technology foraccelerating image processing in electronic know your customer or thelike using distinctive image feature quantities such as SIFT. WO2020/008629 discloses a technology capable of improving accuracy ofimage processing in electronic know your customer or the like.

SUMMARY OF THE INVENTION

In electronic know your customer, a text string such as a name, anaddress, and a birth day described in a target is extracted byperforming text recognition on an image of the original of the targetsuch as a know your customer document. Then, the know your customer of auser is performed based on the extracted text string.

However, even when an illegal image such as an image obtained by imagingor scanning a copy of a target or a falsified or altered image ispresented from a user rather than an image of the original of thetarget, the presented image may not be detected as the illegal image inthe electronic know your customer in some cases.

The present invention has been devised in view of the problems and anobjective of the present invention is to provide a verification system,a verification method, and a program capable of strictly performingelectronic know your customer.

According to an aspect of the present invention, a verification systemincludes: a front image acquisition unit configured to acquire a frontimage indicating a state in which a written surface on which a documentis written in a target is viewed in a front direction; an oblique imageacquisition unit configured to acquire an oblique image indicating astate in which the written surface is viewed in an oblique direction; atransformed image generation unit configured to generate a transformedimage indicating a state in which the target shown in the oblique imageis viewed in the front direction by transforming the oblique image basedon a feature extracted from at least a part of the front image and afeature extracted from at least a part of the oblique image; a regionspecifying unit configured to specify a target region indicating thetarget in the front image based on a feature extracted from a typicalportion of a document written on a predetermined target in a sampleimage indicating a known region of the target and a feature extractedfrom at least a part of the front image; and a verification unitconfigured to determine whether a target shown in the oblique image isthe same as a target shown in the front image by verifying the targetregion in the front image and a region in the transformed imageassociated with the target region.

In the verification system according to an aspect of the presentinvention, the region specifying unit may specify the target regionafter the transformed image generation unit generates the transformedimage.

Alternatively, the transformed image generation unit may generate thetransformed image after the region specifying unit specifies the targetregion.

In the verification system according to an aspect of the presentinvention, the transformed image generation unit may generate thetransformed image by transforming the oblique image based on a featureextracted from the target region specified by the region specifying unitand a feature extracted from at least a part of the oblique image.

The verification system according to an aspect of the present inventionmay further include a thickness determination unit configured todetermine whether a thickness of a target shown in the oblique image isthicker than a predetermined thickness based on the oblique image.

The verification system according to an aspect of the present inventionmay further include a text recognition unit configured to recognize textincluded in a document written in the target region.

According to another aspect of the present invention, a verificationmethod includes: acquiring a front image indicating a state in which awritten surface on which a document is written in a target is viewed ina front direction; acquiring an oblique image indicating a state inwhich the written surface is viewed in an oblique direction; generatinga transformed image indicating a state in which the target shown in theoblique image is viewed in the front direction by transforming theoblique image based on a feature extracted from at least a part of thefront image and a feature extracted from at least a part of the obliqueimage; specifying a target region indicating the target in the frontimage based on a feature extracted from a typical portion of a documentwritten on a predetermined target in a sample image indicating a knownregion of the target and a feature extracted from at least a part of thefront image; and determining whether a target shown in the oblique imageis the same as a target shown in the front image by verifying the targetregion in the front image and a region in the transformed imageassociated with the target region.

According to still another aspect of the present invention, anon-transitory computer readable information storage medium storing aprogram which is to be executed by a computer to execute a programcauses a computer to perform: a procedure of acquiring a front imageindicating a state in which a written surface on which a document iswritten in a target is viewed in a front direction; a procedure ofacquiring an oblique image indicating a state in which the writtensurface is viewed in an oblique direction; a procedure of generating atransformed image indicating a state in which the target shown in theoblique image is viewed in the front direction by transforming theoblique image based on a feature extracted from at least a part of thefront image and a feature extracted from at least a part of the obliqueimage; a procedure of specifying a target region indicating the targetin the front image based on a feature extracted from a typical portionof a document written on a predetermined target in a sample imageindicating a known region of the target and a feature extracted from atleast a part of the front image; and a procedure of determining whethera target shown in the oblique image is the same as a target shown in thefront image by verifying the target region in the front image and aregion in the transformed image associated with the target region.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of an overall configurationof an image processing system according to an embodiment of the presentinvention;

FIG. 2 is a diagram illustrating an example of a state in which a driverlicense is imaged;

FIG. 3 is a diagram illustrating an example of a front image;

FIG. 4 is a diagram illustrating an example of an oblique image;

FIG. 5 is a diagram illustrating an example of extracted text data;

FIG. 6 is a functional block diagram illustrating an example of afunction of a server according to an embodiment of the presentinvention;

FIG. 7 is a diagram illustrating an example of a transformed image;

FIG. 8 is a diagram illustrating an example of a sample image;

FIG. 9 is a diagram illustrating an example of a front image;

FIG. 10 is a diagram illustrating an example of a transformed image;

FIG. 11 is a diagram schematically illustrating an example of learningof a machine learning model; and

FIG. 12 is a flowchart illustrating an example of a flow of a processexecuted in the server according to an embodiment of the presentinvention.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, an embodiment of the present invention will be described indetail with reference to the drawings.

FIG. 1 is a diagram illustrating an example of an overall configurationof an image processing system 1 according to an embodiment of thepresent invention. As illustrated in FIG. 1, the image processing system1 according to the embodiment includes, for example, a server 10 and auser terminal 12. The server 10 and the user terminal 12 are connectedto a computer network 14 such as the Internet. Therefore, the server 10and the user terminal 12 can communicate with each other via thecomputer network 14. In FIG. 1, the number of servers 10 and the numberof user terminals 12 are singular, but may be plural.

The server 10 is a computer system such as a server computer andincludes, for example, a processor 10 a, a storage unit 10 b, and acommunication unit 10 c, as illustrated in FIG. 1.

The processor 10 a is, for example, a program control device such as amicroprocessor that operates in accordance with a program installed inthe server 10. The storage unit 10 b is, for example, a storage elementsuch as a ROM or a RAM, or a solid-state drive (SSD). The storage unit10 b stores a program executed by the processor 10 a, or the like. Thecommunication unit 10 c is, for example, a communication interface forwired communication or wireless communication and exchanges data withthe user terminal 12 via the computer network 14.

The user terminal 12 is a computer that is operated by a user and is,for example, a mobile phone (including a smartphone), a portableinformation terminal (including a tablet computer), or a personalcomputer. As illustrated in FIG. 1, the user terminal 12 includes, forexample, a processor 12 a, a storage unit 12 b, a communication unit 12c, an operation unit 12 d, a display unit 12 e, and an imaging unit 12f.

The processor 12 a is, for example, a program control device such as amicroprocessor that operates in accordance with a program installed inthe user terminal 12. The storage unit 12 b is, for example, a storageelement such as a ROM or a RAM, or a solid-state drive (SSD). Thestorage unit 12 b stores a program executed by the processor 12 a, orthe like. The communication unit 12 c is, for example, a communicationinterface for wired communication or wireless communication andexchanges data with the server 10 via the computer network 14.

The operation unit 12 d is an input device and includes, for example, apointing device such as a touch panel or a mouse, or a keyboard. Theoperation unit 12 d delivers operation content to the processor 12 a.The display unit 12 e is, for example, a liquid crystal display unit oran organic EL display unit. The imaging unit 12 f includes at least onecamera and includes, for example, a CMOS image sensor or a CCD imagesensor. The imaging unit 12 f captures a still image or a moving imageand generates image data. In the embodiment, the imaging unit 12 fincluded in the user terminal 12 will be described. However, the imagingunit 12 f may be provided outside of the user terminal 12.

The program and the data stored in the storage units 10 b and 12 b indescription may be supplied from another computer via a network. Ahardware configuration of the server 10 and the user terminal 12 is notlimited to the foregoing example and various hardware can be applied.For example, the server 10 or the user terminal 12 may include a readingunit (for example, an optical disc drive or a memory card slot) thatreads a computer-readable information storage medium or an input/outputunit (for example, a USB port) that inputs and outputs data to and froman external device. For example, the program or the data stored in theinformation storage medium may be supplied to the server 10 or the userterminal 12 via the reading unit or the input/output unit.

In the embodiment, a user images an image of a target such as a knowyour customer document with the imaging unit 12 f and uploads thecaptured image to the server 10 via the Internet to open a bank account,make an insurance contract, or the like.

The know your customer document may be a document with which the usercan be checked and is, for example, a driver license, an insurancecertificate, a resident card, or a passport. In the followingdescription, the know your customer document according to the embodimentis assumed to be a driver license. For the driver license, there arevarious formats for each nation or district. To facilitate description,a driver license with a fictional format will be exemplified.

FIG. 2 is a diagram illustrating an example of a state in which a driverlicense 20 is imaged. As illustrated in FIG. 2, for example, the useruses the imaging unit 12 f of the user terminal 12 to image the driverlicense 20 on a desk. In the embodiment, when the user images the driverlicense 20 substantially at the front position (right above), a frontimage 22 exemplified in FIG. 3 is captured by the imaging unit 12 f.When the user images the driver license 20 obliquely, an oblique image24 exemplified in FIG. 4 is captured by the imaging unit 12 f. In theembodiment, a resolution of the front image 22 or the oblique image 24is guaranteed to the degree that optical text recognition is possibleand the driver license 20 is focused by the imaging unit 12 f.

The user terminal 12 according to the embodiment uploads the front image22 and the oblique image 24 captured by the imaging unit 12 f to theserver 10.

The server 10 performs electronic know your customer (eKYC) using theuploaded front image 22 and the uploaded oblique image 24.

For example, the server 10 performs optical text recognition on thefront image 22 and extracts text such as a name, an address, and a birthday printed on the target. FIG. 5 is a diagram illustrating an exampleof extracted text data indicating text extracted from the front image 22illustrated in FIG. 3.

The server 10 uses a learned machine learning model to determine whetherthe driver license 20 has some thickness shown in the oblique image 24,here, for example, whether the thickness is thicker than a predeterminedthickness.

The server 10 determines whether a target shown in the front image 22 isthe same as a target shown in the oblique image 24 by verifying thefront image 22 and the oblique image 24.

In the electronic know your customer, an illegal image such as an imageobtained by copying or scanning the driver license 20 or a falsified oraltered image is presented from a user rather than an image of theoriginal of the driver license 20 in some cases. In these cases, in theelectronic know your customer, it may not be detected that the presentedimage is the illegal image.

In the embodiment, the electronic know your customer can be strictlyperformed by performing the electronic know your customer using thefront image 22 and the oblique image 24 obtained by the same target.

Hereinafter, a function of the server 10 and a process executed by theserver 10 according to the embodiment will be further described.

FIG. 6 is a functional block diagram illustrating an example of afunction implemented by the server 10 according to the embodiment. Theserver 10 according to this embodiment does not need to implement all ofthe functions illustrated in FIG. 6, and may also implement otherfunctions than those illustrated in FIG. 6.

As illustrated in FIG. 6, the server 10 according to the embodimentfunctionally includes, for example, an image acquisition unit 30, atransformed image generation unit 32, a sample image storage unit 34, atarget region specifying unit 36, a verification unit 38, a machinelearning model 40, a thickness determination unit 42, and a textrecognition unit 44. The image acquisition unit 30 is implemented mainlyby the communication unit 10 c. The transformed image generation unit32, the target region specifying unit 36, the verification unit 38, thethickness determination unit 42, and the text recognition unit 44 areimplemented mainly by the processor 10 a. The sample image storage unit34 is implemented mainly by the storage unit 10 b. The machine learningmodel 40 is implemented mainly by the processor 10 a and the storageunit 10 b.

The functions given above are implemented in this embodiment by theprocessor 10 a of the server 10 which is a computer by executing aprogram that is installed in the server 10 and includes commandscorresponding to the above-mentioned functions. This program is suppliedto the server 10 via a computer-readable information storage medium suchas an optical disc, a magnetic disk, magnetic tape, a magneto-opticaldisk, or a flash memory, or via a computer network such as the Internet.

The image acquisition unit 30 acquires, for example, a front image 22indicating a state in which a written surface on which a document iswritten in a target such as a know your customer document is viewed in afront direction in the embodiment. The image acquisition unit 30acquires, for example, a front image 22 transmitted from the userterminal 12. The front image 22 according to the embodiment may notnecessarily be an image indicating a state in which the written surfaceis strictly viewed at the front position, and suffices to represent astate in which the written surface is substantially viewed at the frontposition to the degree that optical text recognition is possible.

A document according to the embodiment is assumed to have apredetermined format and a layout is assumed to be determined inadvance. That is, for the document, what is depicted where is assumed tobe known in advance.

The document according to the embodiment is assumed to include a typicalportion and an atypical portion. The typical portion is a portion inwhich content is fixed and is a portion common to other documents. Inother words, the typical portion is a portion in which content is notchanged irrespective of documents and is a portion in which content isnot changed irrespective of users. For example, the typical portion is aformat portion in a document and is a portion in which specific text,signs, figures, enclosing lines, an illustrate, or an image is depicted.The typical portion can also be a portion including information uniqueto a document.

In the example of FIG. 3, a title such as “DRIVER LICENSE” is an exampleof the typical portion. Item names such as “NAME”, “BIRTH DAY”,“ADDRESS”, “DATE”, “EXPIRES”, and “NUMBER” are examples of the typicalportion. A name of country such as “JAPAN” is an example of the typicalportion. A name of an institution such as “Tokyo Metropolitan PublicSafety Commission” is an example of the typical portion. The typicalportion is not limited to the foregoing text. An image indicating thenational flag of Japan in the driver license 20 in FIG. 3 is an exampleof the typical portion. An enclosing line that encloses the foregoingitem name is also an example of the typical portion.

The atypical portion is a portion in which content is not fixed and is aportion in which content is not common to another document. In otherwords, the atypical portion is a portion in which content is changed foreach document and is a portion in which content is changed for eachuser. For example, the atypical portion is a portion other than a formatportion in a document and is a portion in which information such aspersonal information such as identification information or an attributeof a user is printed. The atypical portion can also be a portionincluding information unique to the user.

In the example of FIG. 3, a name “YAMADA TARO” is an example of theatypical portion. A birth day “June 23, 1980” is an example of theatypical portion. An address “1-2-3 ABC City Tokyo” is an example of theatypical portion. An issue date “July 25, 2015” is an example of theatypical portion. An expire date “July 25, 2020” is an example of theatypical portion. A license number “1234 5678 9012” is an example of theatypical portion. The atypical portion is not limited to the foregoingtext. A face photo of the user in the driver license 20 of FIG. 3 isalso an example of the atypical portion. In addition, when informationindicating a physical feature or an ID of the user is included in thedriver license 20, the information is also an example of the atypicalportion.

In the embodiment, the image acquisition unit 30 acquires, for example,the oblique image 24 indicating a state in which the written surface isviewed in an oblique direction. The image acquisition unit 30 acquires,for example, the oblique image 24 transmitted from the user terminal 12.

In the embodiment, the transformed image generation unit 32 generates atransformed image 50 indicating a state in which a target shown in theoblique image 24 is viewed in the front direction, as exemplified inFIG. 7, for example, by transforming the oblique image 24. Thetransformed image generation unit 32 generates the transformed image 50exemplified in FIG. 7, for example, by transforming the oblique image 24based on a feature extracted from at least a part of the front image 22and a feature extracted from at least a part of the oblique image 24.

Here, the extracted feature is, for example, an image feature quantityextracted using an algorithm such as SIFT, SURF, or A-KAZE implementedin OpenCV and includes positional coordinates of a plurality of featurepoints and feature quantities of the feature points. The featurequantities are, for example, numerical values output from theabove-described algorithm and are numerical values obtained bydigitizing features of colors or a distinctive shape of an object.

For example, the transformed image generation unit 32 extracts a featurepoint group from each of the whole front image 22 and the whole obliqueimage 24 by using the above-described algorithm. The transformed imagegeneration unit 32 extracts about tens to thousands or more of featurepoints from the front image 22 and the oblique image 24.

With regard to feature points in the front image 22, the transformedimage generation unit 32 performs matching of the feature point group byspecifying feature points in the oblique image 24 corresponding to thefeature points in the front image 22. In the matching, feature pointswith similar feature quantities may be associated with each other. Thesimilarity of the feature quantities means that the values of thefeature quantities are similar and a difference between the featurequantities is small (for example, the difference is the minimum). Inthis matching, the feature points in the front image 22 are associatedwith the feature points in the oblique image 24.

The transformed image generation unit 32 calculates a transformationmatrix based on a matching result of the feature point group. Thetransformation matrix is calculated so that the position of each featurepoint in the oblique image 24 is close to the position of the featurepoint of a matching part in the front image 22. As a method of acquiringthe transformation matrix, any of various methods can be used. Forexample, a calculation expression of a transformation matrix in affinetransformation, linear transformation, or projection transformation maybe used.

The transformed image generation unit 32 generates the transformed image50 by transforming the oblique image 24 based on the transformationmatrix. As illustrated in FIG. 7, a document shown in the transformedimage 50 is roughly similar to a document shown in the front image 22illustrated in FIG. 3.

The feature point group may be extracted from the entire image, but maybe extracted from a partial region.

Here, a case in which the feature point group is used is exemplifiedherein. The transformed image generation unit 32 may transform theoblique image 24 based on information which is a feature of an image, orinformation other than the feature point group may be used.

A transformation scheme for the oblique image 24 is not limited toaffine transformation, linear transformation, or projectiontransformation. In the transformation of the oblique image 24, rotation,scaling, or movement maybe used. Some or all of affine transformation,linear transformation, projection transformation, rotation, scaling, andmovement may be combined.

The sample image storage unit 34 stores, for example, a sample image 52illustrated in FIG. 8 in the embodiment. The sample image 52 accordingto the embodiment is an image in which there is no distorted curve orsubstantially no distorted curve of a document written on a writtensurface. In other words, the sample image 52 is an image in which adocument is captured in a front direction or in a substantial frontdirection. The front direction is a direction of which an angle formedwith the written surface of the document is perpendicular or may bedirectly opposite. The substantial front direction is a direction ofwhich the angle is substantially perpendicular and is, for example, adirection of which the angle is equal to or greater than 80 degrees. Theformat of the sample image 52 is the same as the format of the frontimage 22. Therefore, a typical portion of the sample image 52 is thesame as a typical portion of the front image 22 and an atypical portionof the sample image 52 is different from an atypical portion of thefront image 22. The sample image 52 may not include atypical portion.That is, the sample image 52 may have only a format portion.

As illustrated in FIG. 8, in the sample image 52, the shape of therounded quadrangular driver license 54 is maintained and the sampleimage 52 has no distortion or substantially no distortion. The directionof the driver license 54 is not shifted and there is no curve orsubstantially no curve. Therefore, text of the sample image 52 is notdistorted or curved and is appropriate for optical text recognition. Forexample, the sample image 52 is prepared in advance by a manager of theimage processing system 1. For example, the manager generates the sampleimage 52 by capturing a target on which a document is written with animaging device or an image reading device such as a scanner andregisters the sample image 52 in the sample image storage unit 34.

The background of the sample image 52 is preferably, for example,monochrome such as black or white.

The sample image 52 according to the embodiment is an image in which apredetermined target (for example, the driver license 54 herein) isshown in a known region. Hereinafter, this region is referred to as asample target region R1. That is, a position, a shape, and a size of thesample target region R1 in the sample image 52 are known in advance. Thesample target region R1 according to the embodiment is a rectangularregion enclosing the driver license 54 shown in the sample image 52.Sample target region data indicating the sample target region R1 isstored in advance in the sample image storage unit 34. The sample targetregion data is, for example, data indicating coordinate values ofvertexes (for example, four vertexes P1, P2, P3, and P4 herein) of thesample target region R1.

The target region specifying unit 36 specifies a region in which thetarget is shown in the front image 22, for example, in the embodiment.As illustrated in FIG. 9, hereinafter, this region is referred to as afront target region R2 in some cases.

The target region specifying unit 36 specifies the front target R2,illustrated in FIG. 9, for example, based on a feature extracted fromthe typical portion of the document written in the target in the sampleimage 52 and a feature extracted from at least a part of the front image22.

Here, the extracted feature is, as described above, an image featurequantity extracted using an algorithm such as SIFT, SURF, or A-KAZEimplemented in OpenCV. The extracted feature includes positionalcoordinates of a plurality of feature points and feature quantities ofthe feature points. The feature quantities are, for example, numericalvalues output from the above-described algorithm and are numericalvalues obtained by digitizing features of colors or a distinctive shapeof an object.

For example, with regard to the feature points of the typical portionextracted from the sample image 52, the target region specifying unit 36matches the feature point group by specifying the feature points in thefront image 22 corresponding to the feature points of the typicalportion. Through this matching, the feature points in the typicalportion of the sample image 52 are associated with the feature points inthe front image 22.

The target region specifying unit 36 specifies the front target regionR2 in the front image 22 corresponding to the sample target region R1based on the matching result and the sample target region data stored inthe sample image storage unit 34. For example, the target regionspecifying unit 36 specifies coordinate values of vertexes (in theexample of FIG. 9, four vertexes P5, P6, P7, and P8) of the front targetregion R2 in the front image 22. The quadrangular front target region R2according to the embodiment may not be rectangular. For example, whenthe front image 22 is not in a state in which the written surface isstrictly viewed at the front position, the front target region R2 is notrectangular.

In the embodiment, the target region specifying unit 36 specifies, forexample, a region (a transformed target region R3 illustrated in FIG.10) in the transformed image 50 associated with the front target regionR2 in the front image 22. In the embodiment, for example, by disposingthe front image 22 and the transformed image 50 on the same coordinatesystem, it is possible to specify the transformed target region R3 inthe transformed image 50 associated with the front target region R2 inthe front image 22. Here, for example, the coordinate values of vertexes(in the example of FIG. 10, four vertexes P9, P10, P11, and P12) of thetransformed target region R3 are specified. The quadrangular transformedtarget region R3 according to the embodiment may not be rectangular asin the front target region R2.

In the embodiment, the verification unit 38 determines whether thetarget shown in the oblique image 24 is the same as the target shown inthe front image 22, for example, by verifying the front target region R2in the front image 22 and the transformed target region R3 in thetransformed image 50.

In the embodiment, for example, the shape and the size of the fronttarget region R2 match the shape and the size of the transformed targetregion R3. Thus, with regard to the pixels in the front target regionR2, and thus the pixels in the transformed target region R3 associatedwith these pixels can be specified. For example, with regard to thepixels in the front target region R2, differences between pixel valuesof the pixels and pixel values of the pixels in the transformed targetregion R3 associated with these pixels are calculated. Then, a sum ofthe differences of the pixel values calculated for the pixels in thefront target region R2 is calculated.

The verification unit 38 determines that the target shown in the obliqueimage 24 is the same as the target shown in the front image 22 when thecalculated sum is less than a predetermined value. Otherwise, theverification unit 38 determines that the target shown in the obliqueimage 24 is not the same as the target shown in the front image 22.

In the embodiment, the verification unit 38 may verify an image of theface of the user shown in the front image 22 and an image of the face ofthe user shown in the transformed image 50 (perform face authentication)in addition to the above-described texture matching. When the calculatedsum of the differences of the pixel values is less than thepredetermined value and the face authentication is successful, it may bedetermined that the target shown in the oblique image 24 is the same asthe target shown in the front image 22. When the calculated sum of thedifferences of the pixel values is not less than the predetermined valueor the face authentication fails, it may be determined that the targetshown in the oblique image 24 is not the same as the target shown in thefront image 22.

Here, when it is determined that the target shown in the oblique image24 is not the same as the target shown in the front image 22, theverification unit 38 may notify the user of a request for re-uploadingthe front image 22 and the oblique image 24.

The machine learning model 40 is, for example, a machine learning modelsuch as a convolutional neural network (CNN) in the embodiment. In theembodiment, for example, it is assumed that the manager of the imageprocessing system 1 performs learning of the machine learning model 40in advance. As illustrated in FIG. 11, for example, in the embodiment, aplurality of pieces of training data including learning input images andteacher data are prepared in advance. The learning input images are, forexample, images obtained by obliquely imaging various objects, such asimages obtained by capturing the original of a target such as a driverlicense in the oblique direction or images obtained by capturing a thinsheet in the oblique direction.

For example, teacher data (for example, teacher data with a value of 1)indicating a positive example is associated with the learning inputimage obtained by capturing the original of the target. Conversely,teacher data (for example, teacher data with a value of 0) indicating anegative example is associated with a learning input image obtained bycapturing an object which is not the original of the target. In thisway, a plurality of pieces of training data including the learning inputimage and teacher data associated with the learning input image aregenerated.

Then, learning of the machine learning model 40 is performed usingoutput data which is an output when the learning input image included inthe training data is input to the machine learning model 40. Here, forexample, differences between output data which is an output uponinputting the learning input image included in the training data to themachine learning model 40 and the teacher data included in the trainingdata may be specified. Supervised learning in which values of parametersof the machine learning model 40 are updated may be performed based onthe specified differences using a scheme such as back propagationmethod.

The machine learning model 40 may not necessarily be the above-describedclassification model and may be a regression model. In this case, valuesof teacher data or output data may indicate the thickness of a target.

In the embodiment, the thickness determination unit 42 determineswhether the thickness of the target shown in the oblique image 24 isthicker than a predetermined thickness based on, for example, theoblique image 24. Here, for example, when it is determined that thetarget shown in the oblique image 24 is the same as the target shown inthe front image 22, the thickness determination unit 42 may perform thedetermination.

Here, for example, the machine learning model 40 is assumed to be theabove-described classification model. In this case, when an output uponinputting the oblique image 24 to the learned machine learning model 40is “1,” it is determined that the thickness of the target shown in theoblique image 24 is thicker than the predetermined thickness. When anoutput upon inputting the oblique image 24 to the learned machinelearning model 40 is “0,” it is determined that the thickness of thetarget shown in the oblique image 24 is thinner than the predeterminedthickness.

For example, the machine learning model 40 is assumed to be theabove-described regression model. In this case, when an output uponinputting the oblique image 24 to the learned machine learning model 40is equal to or greater than a predetermined value, it is determined thatthe thickness of the target shown in the oblique image 24 is thickerthan the predetermined thickness. When the output upon inputting theoblique image 24 to the learned machine learning model 40 is less than apredetermined value, it is determined that the thickness of the targetshown in the oblique image 24 is thinner than the predeterminedthickness.

Here, when it is determined that the thickness of the target shown inthe oblique image 24 is thinner than the predetermined thickness, thethickness determination unit 42 may notify the user of a request forre-uploading the front image 22 and the oblique image 24.

In the embodiment, the text recognition unit 44 recognizes text includedin the document written in the front target region R2 through, forexample, optical text recognition or the like. The text recognition unit44 generates, for example, the extracted text data exemplified in FIG.5.

Here, an example of a flow of a process executed by the server 10according to the embodiment will be described with reference to theflowchart exemplified in FIG. 12.

First, the image acquisition unit 30 receives the front image 22exemplified in FIG. 3 and the oblique image 24 illustrated in FIG. 4from the user terminal 12 (S101).

Then, the transformed image generation unit 32 generates the transformedimage 50 exemplified in FIG. 7 by transforming the oblique image 24based on the front image 22 and the oblique image 24 received in theprocess of S101 (S102).

Then, the target region specifying unit 36 specifies the front targetregion R2 in the front image 22 based on the sample image 52 exemplifiedin FIG. 8, the sample target region data, and the front image 22received in the process of S101 (S103).

Then, the target region specifying unit 36 specifies the transformedtarget region R3 in the transformed image 50 generated in the process ofS102 which corresponds to the front target region R2 specified in theprocess of S103 (S104).

Then, the verification unit 38 performs the verification process todetermine whether the target shown in the oblique image 24 received inthe process of S101 is the same as the target shown in the front image22 received in the process of S101 (S105). Here, for example, thedetermination is performed based on the pixel values of the pixels inthe front target region R2 specified in the process of S103 and thepixel values of the pixels in the transformed target region R3 specifiedin the process of S104.

Then, based on the oblique image 24 received in the process of S101, thethickness determination unit 42 performs a thickness determinationprocess of determining whether the thickness of the target shown in theoblique image 24 is thicker than the predetermined thickness (S106).

Then, the text recognition unit 44 performs a text recognition processof recognizing text included in the document written in the front targetregion R2 specified in the process of S103 through the optical textrecognition or the like to generate the extracted text data exemplifiedin FIG. 5 (S107). Then, the process in the exemplary process ends.

In the process illustrated in FIG. 12, as illustrated in S102 and S103,after the transformed image generation unit 32 generates the transformedimage 50, the target region specifying unit 36 specifies the fronttarget region R2. Here, an order of the process of S102 and the processof S103 may be reversed. The transformed image generation unit 32 maygenerate the transformed image 50 after the target region specifyingunit 36 specifies the front target region R2.

In this case, the transformed image generation unit 32 may generate thetransformed image 50 by transforming the oblique image 24 based on thefeature extracted from the front target region R2 specified by thetarget region specifying unit 36 and the feature extracted from at leasta part of the oblique image 24. In this way, since the number of featurepoints used for the matching is narrowed down than when the featurepoints are extracted from the entire front image 22, the transformedimage 50 can be generated with a processing load less than in theprocess of S102 in FIG. 12.

In the embodiment, as described above, in the electronic know yourcustomer, it is checked that the target shown in the oblique image 24has some thickness and the target shown in the front image 22 is thesame as the target shown in the oblique image 24. In this way, accordingto the embodiment, it is possible to strictly check that the user ownsthe original of the target. As a result, the electronic know yourcustomer can be more strictly performed.

In accordance with images, backgrounds, the sizes of regions in whichtargets are shown, methods of causing ambient light to arrive, colors,and brightness are various. For example, a background on which the frontimage 22 is captured is different from a background on which the obliqueimage 24 is captured in some cases. Therefore, even though the frontimage 22 and the oblique image 24 are simply combined, it may not beaccurately determined whether the target shown in the front image 22 isthe same as the target shown in the oblique image 24.

In the embodiment, as described above, since the front target region R2specified based on the sample image 52 is combined with the transformedtarget region R3, it can be accurately determined whether the targetshown in the front image 22 is the same as the target shown in theoblique image 24.

The present invention is not limited to the above-described embodiments.

For example, role sharing of the server 10 and the user terminal 12 isnot limited to the above description. For example, some or all of thefunctions illustrated in FIG. 6 may be implemented by the user terminal12.

The front image 22 is not necessarily captured by the imaging unit 12 fand may be read by a scanner.

The foregoing specific text string or numeral values and the specifictext strings and numeral values in the drawings are exemplary and thepresent invention is not limited to these text strings or numericalvalues.

While there have been described what are at present considered to becertain embodiments of the invention, it will be understood that variousmodifications maybe made thereto, and it is intended that the appendedclaims cover all such modifications as fall within the true spirit andscope of the invention.

What is claimed is:
 1. A verification system comprising: at least oneprocessor; and at least one memory device storing instructions which,when executed by the at least one processor, cause the at least oneprocessor to perform operations comprising: acquiring a front imageindicating a state in which a written surface on which a document iswritten in a target is viewed in a front direction; acquiring an obliqueimage indicating a state in which the written surface is viewed in anoblique direction; generating a transformed image indicating a state inwhich the target shown in the oblique image is viewed in the frontdirection by transforming the oblique image based on a feature extractedfrom at least a part of the front image and a feature extracted from atleast a part of the oblique image; specifying a target region indicatingthe target in the front image based on a feature extracted from atypical portion of a document written on a predetermined target in asample image indicating a known region of the target and a featureextracted from at least a part of the front image; and determiningwhether a target shown in the oblique image is the same as a targetshown in the front image by verifying the target region in the frontimage and a region in the transformed image associated with the targetregion.
 2. The verification system according to claim 1, whereinspecifying the target region after generating the transformed image. 3.The verification system according to claim 1, wherein generating thetransformed image after specifying the target region.
 4. Theverification system according to claim 3, wherein the generatingcomprises generating the transformed image by transforming the obliqueimage based on a feature extracted from the target region specified bythe specifying and a feature extracted from at least a part of theoblique image.
 5. The verification system according to claim 1, whereinthe operations further comprise: determining whether a thickness of atarget shown in the oblique image is thicker than a predeterminedthickness based on the oblique image.
 6. The verification systemaccording to claim 1, wherein the operations further comprise:recognizing text included in a document written in the target region. 7.A verification method comprising: acquiring a front image indicating astate in which a written surface on which a document is written in atarget is viewed in a front direction; acquiring an oblique imageindicating a state in which the written surface is viewed in an obliquedirection; generating a transformed image indicating a state in whichthe target shown in the oblique image is viewed in the front directionby transforming the oblique image based on a feature extracted from atleast a part of the front image and a feature extracted from at least apart of the oblique image; specifying a target region indicating thetarget in the front image based on a feature extracted from a typicalportion of a document written on a predetermined target in a sampleimage indicating a known region of the target and a feature extractedfrom at least a part of the front image; and determining whether atarget shown in the oblique image is the same as a target shown in thefront image by verifying the target region in the front image and aregion in the transformed image associated with the target region.
 8. Anon-transitory computer readable information storage medium storing aprogram which is to be executed by a computer to execute: a procedure ofacquiring a front image indicating a state in which a written surface onwhich a document is written in a target is viewed in a front direction;a procedure of acquiring an oblique image indicating a state in whichthe written surface is viewed in an oblique direction; a procedure ofgenerating a transformed image indicating a state in which the targetshown in the oblique image is viewed in the front direction bytransforming the oblique image based on a feature extracted from atleast a part of the front image and a feature extracted from at least apart of the oblique image; a procedure of specifying a target regionindicating the target in the front image based on a feature extractedfrom a typical portion of a document written on a predetermined targetin a sample image indicating a known region of the target and a featureextracted from at least a part of the front image; and a procedure ofdetermining whether a target shown in the oblique image is the same as atarget shown in the front image by verifying the target region in thefront image and a region in the transformed image associated with thetarget region.